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Purpose: The purpose of this study was to determine the validity and reliability of the Thai 
version of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU), when 
compared to the diagnoses made by delirium experts. 

Patients and methods: This was a cross-sectional study conducted in both surgical intensive 
care and subintensive care units in Thailand between February- June 2011. Seventy patients 
aged 60 years or older who had been admitted to the units were enrolled into the study within 
the first 48 hours of admission. Each patient was randomly assessed as to whether they had 
delirium by a nurse using the Thai version of the CAM-ICU algorithm (Thai CAM-ICU) 
or by a delirium expert using the Diagnostic and Statistical Manual of Mental Disorders, 
Fourth Edition, Text Revision. 

Results: The prevalence of delirium was found to be 18.6% (n=13) by the delirium experts. 
The sensitivity of the Thai CAM-ICU's algorithms was found to be 92.3% (95% confidence 
interval [CI] =64.0%-99.8%), while the specificity was 94.7% (95% CI =85.4%-98.9%). 
The instrument displayed good interrater reliability (Cohen's K =0.81 ; 95% CI =0.64-0.99). The 
time taken to complete the Thai CAM-ICU was 1 minute (interquatile range, 1-2 minutes). 
Conclusion: The Thai CAM-ICU demonstrated good validity, reliability, and ease of use 
when diagnosing delirium in a surgical intensive care unit setting. The use of this diagnostic 
tool should be encouraged for daily, routine use, so as to promote the early detection of delirium 
and its rapid treatment. 

Keywords: delirium, surgical intensive care unit, Confusion Assessment Method for the 
intensive care unit, validity, reliability 

Introduction 

Delirium is a disturbance in a person's consciousness and a change in cognition that 
develops over a short period and usually fluctuates over time. 1 Delirium is a complica- 
tion found commonly in intensive care units (ICUs); its incidence in ICUs varies from 
between 20%-80%, depending on the types of screening tool used and the populations 
involved. 2-15 Patients who suffer from delirium incur higher medical costs, 16 require 
longer hospital stays, 4,617 and have higher mortality rates than those who do not. 4,6,1415 
In addition, delirium is associated with long-term cognitive impairment. 17 

A diagnosis of delirium is quite a challenge for nonpsychiatric medical personnel. 
Many critically ill patients often require mechanical ventilation, meaning that the 
symptoms normally presented by delirious patients cannot be diagnosed using standard 
tools normally used with patients who can communicate verbally. Delirium can go 
unrecognized when diagnosis relies on clinical presentation alone; 13 therefore, delirium 
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screening tools are required in order to detect symptoms early 
and provide for immediate treatment. 

In Thailand, delirium screening is not routinely per- 
formed by medical staff in ICUs due to a lack of available, 
standardized tools, meaning that most diagnoses rely on 
clinical symptoms only. The Confusion Assessment Method 
algorithm (CAM algorithm) has been available for clinical 
diagnosis of delirium by nurses or physicians who are non- 
delirium experts; however, its poor reliability limits the use 
of this tool among patients in the ICU. 718 The Confusion 
Assessment Method for the Intensive Care Unit (C AM-ICU) 
is a highly sensitive and reliable tool for the diagnosis of 
delirium in the ICU. 4 The CAM-ICU has been translated into 
several languages, including German, 3 Dutch, 19 Swedish, 20 
Greek, 8 Portuguese, 10 Spanish, 21 Chinese, 22 and Korean. 9 
A Thai version of the CAM-ICU (Thai CAM-ICU) has 
already been produced, however, it has not yet been 
validated. The objective of this study was to determine 
the validity and reliability of the Thai CAM-ICU when 
compared to a standard reference diagnostic criteria, as 
used by delirium experts, in a surgical intensive care unit 
(SICU). 

Materials and methods 

This cross-sectional study was approved ethically by the 
independent Institutional Review Board at the Faculty 
of Medicine, Chiang Mai University. Three well-trained, 
critical-care study nurses informed and obtained consent 
from the participants or their surrogate decision-makers prior 
to the assessment. The clinicaltrials.gov registration number 
of this study is NCT 01267630. 

Participants and procedures 

Based on the prevalence of delirium within ICUs (50%) 3 
and the acceptable levels of sensitivity and specificity for 
the CAM-ICU (about 90%), 23 70 participants were recruited. 
Those recruited were elderly people 60 years of age and 
older, who had been admitted within the previous 48 hours 
to a 14-bed SICU and sub-SICU at Maharaj Nakorn Chiang 
Mai Hospital. Recruitment took place from February 2011 
to June 20 1 1 . The participants were excluded if they had 
severe visual or auditory impairment, were likely to pass 
away within 24 hours, had a poor command of Thai, were 
unwilling to participate, or were comatose. 

A research nurse and a delirium expert were responsible 
for assessing selected admissions within the first 48 hours of 
arrival at the ICU between the times of 8:30 am and 5 pm on 
weekdays. The nurse carried out the assessments using the 



Thai CAM-ICU, while the delirium expert used criteria from 
the Diagnostic and Statistical Manual of Mental Disorders, 
Fourth Edition, Text Revision (DSM-IV-TR). The assess- 
ments were randomly assigned at interrater intervals of no 
more than 30 minutes. Each participant was evaluated twice, 
once by a nurse (SM or ND) and once by a delirium expert 
(NW or PD), independently. 

Before screening for delirium, each participant was 
assessed using the Richmond Agitation and Sedation Scale 
(RASS) to determine the level of sedation. 24 RASS is a ten- 
point scale that ranges from +4 (combative) to -5 (deeply 
sedated or unresponsive). If participants are alert, agitated, or 
combative, they are given a score of +4 to 0. If participants 
are drowsy or awake only briefly, but are able to respond to 
verbal stimuli, they are given a score of-1 to -3. If patients 
do not respond to verbal stimuli at all and do not respond to 
any physical stimuli, they are scored -4 to -5. In this study, 
all participants needed to score -3 or better using RASS 
in order to take part. A positive mental status change was 
defined as one where a participant showed evidence of an 
acute onset of mental status change when compared to the 
baseline upon admission, or any fluctuation in their RASS 
score, within 24 hours of being initially assessed. 

The Attention Screening Examination (ASE) tool was 
used to assess inattention. The ASE consists of visual and 
auditory components through the recognition of ten letters. 
Inattention was diagnosed when patients scored less than eight 
on any component. To test disorganized thinking, participants 
were asked to answer a set of four questions correctly. An 
alteration of consciousness was rated when the RASS score 
was not zero. 

Development of the Thai version 
of the CAM-ICU 

The Thai version of the CAM-ICU was developed by the 
third author (SM) of this paper, following the "Principles of 
Good Practice for the Translation and Cultural Adaptation 
Process for Patient-Reported Outcomes Measures." 25 First, 
SM translated the CAM-ICU into Thai. Then a bilingual 
translator, who had never been exposed to the original 
CAM-ICU, did a back translation. Both translators reached 
a consensus after comparing and adapting the instrument so 
that its meaning corresponded to Thai culture. 

The CAM-ICU tool assesses four features: (1) the acute 
onset of mental status changes, or a fluctuating course, 
(2) inattention, (3) disorganized thinking, and (4) altered 
levels of consciousness. Delirium is diagnosed when a par- 
ticipant manifests features 1 and 2, plus either feature 3 or 4. 
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The training manual for the Thai CAM-ICU is currently 
available from the website www.icudelirium.org , 

CAM-ICU training and interrater 
reliability 

To ensure the reliability of the CAM-ICU, two critical care 
nurses (SM and ND) who are authors of this study performed 
Thai CAM-ICU assessments on ten patients. A gold standard 
evaluation was also performed by two delirium experts (NW 
and PD) on another ten patients using DSM-IV-TR criteria. 
There was found to be perfect agreement in each group of 
researchers (Cohen's k =1 for both). 

Statistical analysis 

Demographic variables are presented as mean + standard 
deviation for continuous data, and frequency and percentage 
for categorical data. The performance of the Thai version of 
CAM-ICU algorithms was calculated by using sensitivity, 
specificity, and predictive values (positive predictive value 
and negative predictive value). The likelihood ratio of the test 
(sensitivity/ 1 - specificity) including 95% confidence inter- 
vals (CIs) were estimated. A receiver-operating characteristic 



(ROC) curve was presented to demonstrate the performance 
of the Thai version of the CAM-ICU. 26 Cohen's k coeffi- 
cient was used to demonstrate interrater reliability between 
critical care nurses and delirium experts. All statistics were 
analyzed using STATA 1 1 .0 (StataCorp LP, College Station, 
TX, USA). A P-value <0.05 was considered statisticaly 
significant. 

Results 

Patient characteristics 

Over a 5-month period, 92 patients were enrolled in the 
study. Three were excluded because they did not want to 
participate, or their relatives did not want them to. Seventeen 
patients were discharged and two patients died before an 
assessment could take place. Figure 1 shows the details of 
the study flow. 

A total of 70 patients completed the study. The baseline 
patient characteristics are presented in Table 1 . The majority of 
the study patients were male (64.3%) and most had also gradu- 
ated from primary school (64.7%). The percentage of delirious 
patients who required mechanical ventilation was 22.7% and 
the mortality rate during hospitalization was 18.2%. 



Patients meeting inclusion criteria (n=92) 



Exclusion (n=3) 

- Patients or relatives refusing to participate (n=3) 



Patients participating in the study and completing the 
assessment (n=89) 



Exclusion (n=19) 

- Patients discharged before interview (n=17) 

- Patients who died before interview (n=2) 



▼ 

Patients included in the analysis (n=70) 



Figure I Study patient selection process. 
Abbreviation: n, number. 
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Table I Baseline characteristics of the study participants 



Characteristics 


Frequency (n=70) 


Age (years), median (interquartile range) 


71.5 (62.0-78.0) 


Male, n (%) 


45 (64.3%) 


Body Mass Index (kg/m 2 ), mean ± SD 


20.54±3.32 


Level of education, n (%) 




Primary school 


44 (64.7%) 


Secondary school 


8 (1 1.8%) 


University 


6 (8.8%) 


Unknown 


10 (14.7%) 


APACHE II, median (interquartile range) 


13 (10-17) 


Glasgow Coma Score, mean ± SD 


14.3+1.29 


Delirium, n (%) by DSM-IV-TR 


13 (18.6%) 


Delirium, n (%) by CAM-ICU 


12 (17.1%) 


Mechanically ventilated, n (%) 




Delirium 


5 (22.7%) 


Causes of ICU admission, n (%) 




Sepsis 


9 (12.8%) 


Respiratory failure 


7 (10.0%) 


Myocardial infarction/congestive 


5 (7.1%) 


heart failure 




Cardiac arrest/shock 


3 (4.3%) 


Hepatic or renal failure 


3 (4.3%) 


Other causes 


43 (61.4%) 



Table 2 Comparison of delirium assessments carried out by the 
critical care study nurses and the delirium experts 



Hospital stay (days), median (interquartile range) 17(1 1-25) 

Notes: Data are presented as number and percent, mean ± SD, or median 
(25th-75th percentiles). 

Abbreviations: APACHE II, Acute Physiologic and Chronic Health Evaluation II; 
CAM-ICU, The Confusion Assessment Method for the Intensive Care Unit; 
DSM-IV-TR, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, 
Text Revision; n, number; SD, standard deviation; ICU, intensive care unit. 



Validity and interrater reliability 
of the Thai CAM-ICU 

The assessments made by the two critical care nurses and 
the delirium experts are shown in Table 2. The prevalence 
of delirium in SICU patients during the first 48 hours 
after admission was found to be 18.6%. The sensitiv- 
ity of the Thai CAM-ICU's algorithms was 92.3% (95% 
CI =64.0%-99.8%), while their specificity was 94.7% (95% 
CI =85.4%-98.9%). The positive and negative predictive val- 
ues were 80.0% (95% CI =51.9%-95.7%) and 98.2% (95% 
CI =90.3%-100.0%), respectively. The likelihood ratio of 
Thai CAM-ICU was 17.5 (95% CI =5.8-53.3). The results of 
the ROC analysis showed the area under the curve was 0.93 
(95% CI =0.85-1.00). Cohen's k between the study nurses 
and the delirium experts was 0.81 (95% CI =0.64-0.99), with 
a P- value of <0.001. 

Table 3 presents the results of previous CAM-ICU 
validations, as shown in the literature. In this study, 
four CAM-ICU rating misclassifications were made by the 
two critical care nurses, while three false-positive assess- 
ments and one false negative assessment were identified. 



Delirium by CAM-ICU (nurses) 


Delirium 


No delirium 


Delirium by DSM-IV-TR (experts) 






Delirium 


12 


3 


No delirium 


1 


54 



Abbreviations: CAM-ICU. Confusion Assessment Method for the Intensive 
Care Unit; DSM-IV-TR, Diagnostic and Statistical Manual of Mental Disorders, 
Fourth Edition, Text Revision. 



Among the three patients who gave false positives, two had 
hypovolemic shock and one developed septic shock during 
the ICU admission. One patient who gave a false negative 
was diagnosed with delirium plus a hypoalert condition by 
a delirium expert. 

Time spent by nurses using the Thai 
CAM-ICU algorithms 

The average time taken by the nurses to perform the Thai 
CAM-ICU algorithm was significantly less than the time 
taken by the psychiatric staff to do the full psychiatric 
assessments using DSM-IV-TR (1.59+0.86 minutes versus 
8.48+4.8 1 minutes) (P<0.001). The minimum and maximum 
times used to calculate the Thai CAM-ICU algorithms were 
0.5 minutes and 5 minutes, while the minimum and maximum 
times used for the DSM-IV-TR calculations were 2 minutes 
and 25 minutes. 

Discussion 

This study demonstrated that the Thai version of CAM- ICU 
has a high validity and reliability, and that it takes less time to 
complete than the full psychiatric assessments. The validity of 
the Thai CAM-ICU's algorithms is comparable to the original 
CAM-ICU developed by Ely et al 4 showing sensitivities of 
between 95%- 100%, and specificities of between 89%-93%. 
Previous studies have validated the CAM-ICU in different 
settings, including in an SICU, where Guenther et al 3 reported 
the validity of the CAM-ICU Flowsheet in a SICU. In their 
study, the CAM-ICU Flowsheet was modified from the 
original, which meant changing the numbering for Features 
3 and 4 to allow rapid assessments to take place. Their 
results show a high sensitivity, specificity, and reliability 
(88%-92% versus 100%, Cohen's k =0.96). 3 In this study, 
and when used by the two critical care nurses, the sensitivity 
and the specificity of the Thai CAM-ICU algorithm were 
92.3% and 94.7%, respectively. The Thai CAM-ICU 
algorithms revealed a high diagnostic value when taking 
into account the value of the likelihood ratio (LLR) of the 
test when given by the two nurses (LLR =11.8 and 30.0) 
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and the area under the ROC. When comparing the perfor- 
mance of the test, the LLR of the test was superior to the 
positive and negative predictive values and it was not affected 
by the prevalence of disease. 27 

In addition, the area under the ROC confirmed the power 
of the Thai CAM-ICU algorithm in terms of discriminating 
between delirious and nondelirious patients. 26 The present 
study demonstrated that the Thai CAM-ICU had a higher 
sensitivity and specificity than other validated versions 
of the CAM-ICU algorithms, including the Portuguese 10 
(sensitivity =72.5% and specificity =96.2%), Korean 9 
(sensitivity =77.4% for rater 1 and 89.8% for rater 2 and 
specificity =72.4% for rater 1 and 75.8% for rater 2), and 
Greek versions 8 (sensitivity =95.2% and specificity =85.0%). 
In addition, the validity of the Thai version of CAM-ICU was 
comparable to the Chinese version. 22 A good agreement was 
shown between the two critical care nurses when diagnos- 
ing for delirium (Cohen's k =0.82). The level of agreement 
between the two sets of raters (critical care nurses and psy- 
chiatrists) was comparable to that found with the Korean 
version 9 (Cohen's k=0. 81 and 0.75, respectively), but lower 
than the Greek and Spanish versions. 8,21 

The present study revealed some misclassifications 
within the Thai CAM-ICU ratings (three false positives 
and one false negative). In the three cases of patients show- 
ing false positives with the Thai CAM-ICU, delirium was 
diagnosed based on a positive test for Features 1, 2, and 
4. For the patient with a false negative, a "nondelirious" 
state was diagnosed due to a positive result for Feature 1 
and a negative result for Feature 2, while the psychiatrist 
gave positive results for all the reference standard domains. 
There are two possible explanations for these misclassifica- 
tions. First, the use of sedative drugs may have impaired 
participants' consciousness levels, which would have 
increased the chances of delirium being diagnosed by the 
CAM-ICU. 28 Haenggi et al 28 recommended that sedation 
should be discontinued, and the level of sedation should 
be re-evaluated before any assessment for delirium was 
made. One previous study demonstrated that a patient's 
level of education may have an influence on the results for 
Feature 2: ASE. 22 

The present study found that 64.7% of patients had 
graduated from primary school. Wang et al 22 concluded that 
the reliability of the ASE test is lower for illiterate patients 
(Cohen's k =0.61) and patients with only a primary school 
education (Cohen's k =0.70). They stated that giving an 
explanation of the test methods before doing the ASE audi- 
tory evaluation is very important for illiterate patients. Use of 



Clinical Interventions in Aging 2014:9 



submit your manuscript 



883 



Dovepress 



Pipanmekaporn et al 



Dovepress 



the Thai CAM-ICU in routine, clinical practice will improve 
the accuracy of delirium diagnosis for critically ill patients, 
and particularly in ICU settings, providing a standardized 
tool and serving as a benchmark when comparing against 
CAM-ICU tools from other countries. 29 

This study had a number of strengths. First, we compared 
the delirium diagnosis performance of the Thai CAM-ICU 
against a standard reference used by psychiatrists, and sec- 
ond, evaluations were carried out by both critical care nurses 
and delirium experts in order to reduce individual variability 
during the assessment process. Also, the evaluations carried 
out by the two sets of raters (study nurses and psychiatrists) 
on the same patient were never more than 30 minutes apart. 
This approach reduced the risk of variability in terms of 
diagnosis due to fluctuations in the condition. 

However, the study also had a few limitations. First, 
the two nurses used in the study were not able to evaluate 
all the patients over the study period, due to the manpower 
situation in the SICU. As a result, we could not fully demon- 
strate the differences between the nurses' and psychiatrists' 
assessments. However, the authors did perform an interrater 
reliability assessment of the two critical care nurses, until 
perfect agreement had been achieved, before recruiting the 
study patients. Next, this study did not demonstrate motoric 
subtypes of delirium, which can be categorized by the level 
of RASS found. The presence of delirium symptoms is more 
important than the diagnosis of delirium, because patients with 
different subtypes of delirium, such as hypoactive or hyperac- 
tive delirium, require different treatments and prognoses. 

Conclusion 

The Thai CAM-ICU demonstrated good validity and reli- 
ability, and proved easy to use when diagnosing delirium in a 
surgical intensive care unit setting. The use of this diagnostic 
tool in daily practice should therefore be promoted, in order to 
facilitate the early detection and rapid treatment of delirium. 
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